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EXAMINER'S ANSWER 



This is in response to the appeal brief filed 12/07/2009 appealing from the Office action mailed 
09/04/2009. 




(1) Real Party in Interest 

A statement identifying by name the real party in interest is contained in the brief. 
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(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial proceedings which 
will directly affect or be directly affected by or have a bearing on the Board's decision in the pending 
appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection contained in the brief 
is correct. 

(5) Summary of Claimed Subject Matter 

The summary of claimed subject matter contained in the brief is correct. 

(6) Grounds of Rejection to be Reviewed on Appeal 

The appellant's statement of the grounds of rejection to be reviewed on appeal is correct. 

(7) Claims Appendix 

The copy of the appealed claims contained in the Appendix to the brief is correct. 



(8) Evidence Relied Upon 

6573907 Madrane 



06-2003 



Application/Control Number: 10/589,613 Page 3 

Art Unit: 2175 

(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 

Claims 1-21 and 23-28 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Madrane (6573907). 

Re claim 1, Madrane discloses a data handling device comprising a display (column 2 
lines 1-2 for example) for displaying representations of the media objects (see abstract, root 
images for example), data storage means (see column 20 lines 24-26 for example) for allocating 
metadata tags (see column 13 line 66 to column 14 line 1, information designating these objects 
as "extractable" for example) to the media objects, an input device (interface viewer unit for 
example) comprising means to allow (e.g., not prevent) a representation of a selected media 
object (e.g., area of interest) to be selectively moved by a user into a region of the display (e.g., 
the area of interest is moved until it takes up the whole display, when moving up close) 
representing a selected set of metadata tags (e.g., when moving up close, the display represents 
the area of interest, which includes metadata tags) (see figurers 17-18; col. 12, 1. 57 to col. 13, 1. 
15), and user activated means for causing the selected set of metadata tags to be added to those 
allocated to the selected media object in the data storage means (see column 13 line 66 to column 
14 line 3; column 30, 1. 5-17) (e.g., a designer may designate an area of interest of an image to 
be extractable, and to be bounded and associated with a URL). 

Re claim 2, Madrane discloses a data handling device, configured to allow a user to 
generate additional metadata tags (see column 15, lines 18-20, additional text for example) 
having new values, such that the media objects may be further categorized. 
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Re claim 3, Madrane discloses a data handling device, configured to allow a user to 
obtain a view of media objects to which one or more of a predetermined plurality of metadata 
tags have been added (see figure 43 for example). 

Re claim 4, Madrane discloses a data handling device, configured to allow a user to 
obtain a view of media objects to which each of a predetermined plurality of metadata tags have 
been added (see figure 35 for example). 

Re claim 5, Madrane discloses a data handling device, wherein means are provided to 
provide user control of the maximum number of metadata tag sets to be displayed (define how 
many, see column 7 line27 for example). 

Re claim 6, Madrane discloses a data handling device, in which representations of the 
media objects are capable of being moved between regions of the display area representing sets 
of metadata tags having pre-defined values (identify which are "hot objects", see column 7 lines 
30-31 for example). 

Re claim 7, Madrane discloses a data handling device, comprising means for removing a 
representation of a selected media object from one display area and adding it to a second area, 
thereby applying the metadata tag set associated with the second area to the selected media 
object in place of the set of metadata tags associated with the first area (see column 7 lines 30-33 
for example). 

Re claim 8, Madrane discloses a data handling device wherein a representation of a 
media object selected from a display area associated with a first metadata tag set applied to the 
media object may remain there whilst a copy of the selected media object is placed in a second 
area of the display area, thereby applying the metadata tag set associated with the second area to 
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the media object in addition to the set associated with the first area (see column 20 lines 45-50 
for example). 

Re claim 9, Madrane discloses a data handling device, providing means for indicating the 
number of media objects associated with a given set of metadata tags (column 7 lines 25-3 1 for 
example). 

Re claim 10, Madrane discloses a data handling device, providing means for indicating 
the number of metadata tags associated with one or more media objects (define how 
many. . .what image information, see column 7 lines 25-37 for example). 

Re claim 11, Madrane discloses a data handling device, providing means for identifying 
media objects to which no metadata tags have been applied by providing a display area 
representing an empty set (identify which objects in the scene are "hot objects", see column 7 
lines 30-34 for example). 

Re claim 12, Madrane discloses a data handling device, providing means for selecting a 
subset of the media objects for allocating a given set of metadata tags ("hot objects" and what 
image information will be displayed, see column 7 lines 30-34 for example). 

Re claim 13, Madrane discloses a data handling device, providing means for making the 
size of the display area allocated to each set of metadata tags proportional to the number of 
media objects portrayed therein (see column 5 lines 34-40 for example). 

Re claim 14, Madrane discloses a computer program or suite of computer programs for 
use with one or more computers to provide any of the apparatus as set out in claim 1 (see column 
2 line 10 for example). 
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Re claim 15, Madrane discloses a method of organizing and storing media objects for 
subsequent retrieval, the media objects being represented in a display, wherein in which 
metadata tags are applied to the media objects by selecting an individual media object from the 
display, and causing a set of metadata tags to be added to the selected media object by selectively 
placing a representation of the selected media object in a region of the display selected to 
represent the set of metadata tags to be added (see column 7 lines 25-35 for example). 

Re claim 16, Madrane discloses a method, in which a user may generate additional 
metadata tags having new values, such that the media objects may be further categorized (see 
column 15 lines 17-20 for example). 

Re claim 17, Madrane discloses a method, wherein a view is provided of media objects to 
which one or more of a predetermined plurality of metadata tags have been added (see column 
15 lines 17-20, see figure 43 for example). 

Re claim 18, Madrane discloses a method, wherein a view is provided of media objects to 
which each of a predetermined plurality of metadata tags have been added (see figure 35 for 
example). 

Re claim 19, Madrane discloses a method, wherein provision is made to control the 
maximum number of categories to be displayed (define how many, see column 7 line 27 for 
example). 

Re claim 20, Madrane discloses a method, in which representations of the media objects 
are moved between regions of the display area representing sets of metadata tags having pre- 
defined values (see column 7 lines 30-31 for example). 
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Re claim 21, Madrane discloses a method, wherein a representation of a media object is 
selected from a first display area associated with a first metadata tag set, and a copy of the 
selected representation is placed in a second area of the display whilst the original representation 
remains in the first area, thereby applying the metadata tag set associated with the second area to 
the media object, in addition to the set associated with the first area (see column 20 lines 45-50 
for example). 

Re claim 23, Madrane discloses a method, wherein the number of media objects 
associated with a given set of metadata tags is indicated (see column 7 lines 25-31 for example). 

Re claim 24, Madrane discloses a method, wherein the number of metadata tags 
associated with one or more media objects is indicated (see column 7 lines 25-37 for example). 

Re claim 25, Madrane discloses a method, wherein media objects to which no metadata 
tags have been applied are identified by providing a display area representing an empty set (see 
column 7 lines 25-37 for example). 

Re claim 26, Madrane discloses a method, wherein a subset of the media objects may be 
selected for allocation of a given set of metadata tags (see column 7 lines 25-35 for example). 

Re claim 27, Madrane discloses a method, wherein the size of the display area allocated 
to each set of metadata tags is proportional to the number of media objects portrayed therein (see 
column 5 lines 34-40 for example). 

Re claim 28, Madrane discloses a computer program or suite of computer programs for 
use with one or more computers to provide the method of claim 15 (see column 2 line 10 for 
example). 
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Claim 22 is rejected under 35 U.S.C. 103(a) as being unpatentable over Madrane 
Re claim 22, Madrane substantially discloses a method as set forth in claim 20 above. 
Madrane does not explicitly disclose a representation of a selected media object may be removed 
from a first display area associated with one metadata tag set when added to a second display 
area, thereby applying the set of metadata tags associated with the second display area to the 
selected media item in place of the set of metadata tags associated with the firs display area. 
Madrane teaches of applying the set of metadata tags associated with the second display area to 
the selected media item in place of the set of metadata tags associated with the firs display area 
when added to a second display area. Deleting functions are well known. It would have been an 
obvious matter of design choice to have a representation of a selected media object may be 
removed from a first display area associated with one metadata tag set when added to a second 
display area, since such a modification would have involved the mere application of a known 
technique to a piece of prior art ready for improvement. Where a claimed improvement on a 
device or apparatus is no more than "the simple substitution of one known element for another or 
the mere application of a known technique to a piece of prior art ready for improvement," the 
claim is unpatentable under 35 U.S.C. 103(a). Ex Parte Smith, 83 USPQ.2d 1509,1518-19 
(BPAI, 2007) (citing KSR v. Teleflex, 127 S.Ct. 1727, 1740, 82 USPQ2d 1385, 1396 (2007)). 
Accordingly Applicant claims a combination that only unites old elements with no change in the 
respective functions of those old elements, and the combination of those elements yields 
predictable results; absent evidence that the modifications necessary to effect the combination of 
elements is uniquely challenging or difficult for one of ordinary skill in the art, the claim is 
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unpatentable as obvious under 35 U.S. C. 103(a). Ex Parte Smith, 83 USPQ.2d at 1518-19 
(BPAI, 2007) (citing KSR, 127 S.Ct. at 1740, 82 USPQ2d atl396. 



(10) Response to Argument 



Appellant's arguments have been fully considered but are not persuasive. Examiner reiterates 
that references to specific columns, figures or lines should not be limiting in any way. The entire reference 
provides disclosure related to the claimed invention. 

Appellant argues: 

1) First, Madrane does not teach displaying on the display various regions representing various 
values of the metadata tags. Instead, in Madrane, the metadata tags are located in the various interface 
data files FDIi 10, associated with respective video sequences, which can be "root images", i.e., a 
collection of selective video frames from the entire video sequence. Fig. 2 in Madrane shows that the 
video sequences 8, the respective metadata files 10, and the video interface application program 1 1, are 
recorded on the CD-ROM (col. 1 1 , lines 62-67). None of the frames of the various root images shown in 
Figs. 4 and 17-18 shows regions representing selected sets of metadata tags. Instead, what is displayed 
in the various frames is merely scenes of the video sequence, without any information related to 
metadata values, let alone metadata values selected by the user. In Madrane, the metadata information is 
stored in conventional data file format, including a header and the information related to the respective 
video sequences, i.e., identification of the associated video sequence, identification of basic frames, 
identities and location of extractable objects, etc (page 15, penultimate paragraph). 

Examiner disagrees. 

As to 1), Madrane (fig. 7B; col. 13, 1.60 to col. 14, 1.15) teaches a designer designating a video 
of a person walking from left to right as interesting, and a user clicking on said person walking causing all 
extraneous portions of the displayed frames to disappear, and the person walking appearing to move 
from the left side of the screen to the right side of the screen. Further, Madrane (see fig. 6B) teaches a 
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designer designating an object in a display as interesting, placing the object of interest at the center of the 
screen and zooming in on it, while other distant objects are omitted. A user would then be enabled to click 
on the image designated as interesting and cause the image to be zoomed in while other distant object 
are omitted from the display. Thus, Madrane clearly teaches various display regions (e.g., the regions 
associated with the image of the person walking, or the image to be zoomed) representing various values 
of the metadata tags (e.g., the video images are tagged as interesting, and metadata associated with the 
tags direct how images would be displayed if a user were to click on them). 

Appellant argues: 

2) Even assuming arguendo that it were accepted, the center of the screen (to which the object of 
the interest may be placed by the designer as alleged by the Examiner) does not represent a selected set 
of metadata tags. Madrane therefore fails to disclose a region of the display representing a selected set of 
metadata tags as claimed (page 15, last paragraph). 

Examiner disagrees. 

As to 2), as stated above in 1 ), when a designer designates an object as interesting, places an 
object in the center of the screen, and then zooms on it, the designer adds metadata to the designated 
(i.e., tagged) image that enables a user to click on the image to enlarge it. Thus, Madrane clearly teaches 
a region of the display representing a selected set of metadata tags as claimed. 

Appellant argues: 

3) Moreover, unlike the Examiner's assertion, Madrane does not disclose selectively moving a 
representation of a selected media object to a region of the display. Instead, in Madrane, the user 
designates a position on the screen as a viewing position relative to the displayed image (by clicking with 
the computer mouse). This causes a different sequence of video frames to be displayed (depending on 
the point of view chosen by the user), see, for example, Figs. 5A-5C. However, this is not the same as 
moving the media object (which is the root image) around on the display, as required by claims 1 and 15. 
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The root image, i.e., the cuboid comprising the various frames, remains in the same location, with only 
the flow of the constituent frames changing (page 16, antepenultimate paragraph). 
Examiner disagrees. 

As to 3), as acknowledged by Appellant, Madrane teaches enabling a user to designate a viewing 
position to view a root image comprising various frames, with the result that a different sequence of video 
frames is displayed, different than the original flow of the root image comprising various frames. As one of 
ordinary skill in the art would understand, this is the same as applying a function to the root images to 
enable a user, through the applied functions, to specify a viewing position, with the result that the root 
images are displayed differently than if no function were applied. Thus, Madrane does teach selectively 
moving a representation of a selected media object to a region of the display. 



Appellant argues: 

4) Even if one considers one or more of the objects appearing in the various video frames as the 
claimed media objects (for example, the two people in Figs. 7A, 7B), the user cannot selectively move 
those objects, instead, the user can only watch the flow of the sequence of the video frames that include 
those objects. The language of claim 1 specifically recites "allow a representation of a selected media 
object to be selectively moved by a user." 

Examiner disagrees. 

As to 4), one of ordinary skill in the art would recognize that it is the same to "selectively move 
objects" than to "watch a flow of a sequence of the video frames that include those objects" as a user 
designates viewing positions of a root image that include those objects. 
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Appellant argues: 

5) The Examiner referred to the "zooming" of the root image (by selecting a designation point 
close to the root image displayed on the screen) as reading on the claimed moving of the selected media 
object. However, as mentioned above, the root image does not move. What happens with the "zooming" 
is that additional frames are displayed, so that a more detailed version of the video sequence is shown, 
"when the user designates a viewing position close up to the interactive video icon, the image information 
in the area of interest should be enriched. This is achieved by including, in the displayed image, image 
data relating to additional video frames besides the basic video frames", see col. 12, line 67 to col. 13, 
line 4 in Madrane. In other words, a selected portion of the video icon is not moved, instead additional 
frames in proximity to the selected portion of the video icon are displayed (page 16, last paragraph). 

Examiner disagrees. 

As to 5), one of ordinary skill in the art would readily recognize that both the instant invention and 
Madrane behave similarly, in that what is displayed is an illusion of a selected image that appears to 
move across the screen. In both the instant application and Madrane, the illusion of a selected image 
moving across the screen is generated by "additional frames that are displayed, so that a more detailed 
version of the video sequence is shown" (e.g., in the case of a zoom function). Appellant readily 
acknowledges that Madrane teaches the additional frames. 



Appellant argues: 

6) In the Response to Arguments section of the final Office Action of September 4, 2009, the 
Examiner stated "Thus, clearly, a designer designating the person walking as interesting causes 
metadata tags to be added to a media object based on where the object is placed in the display", see 
bottom of p. 10 of the Office Action. First, in Madrane, it is the designer that sets values for the metadata 
tags, not the user ("The designer of the interface has decided that the two people are objects that may be 
of interest to the end user. Accordingly, he has included, in the interface data file, information designating 
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these objects as "extractable", see col. 13, line 64 to col. 14, line 1). Second, no metadata tags are added 
to the object, i.e., the two people walking toward each other, instead, the various video frames in the 
sequence of frames are altered so that only the two objects are displayed. However, this action is caused 
because of the metadata tags associated with those objects previously. As the two people are shown in 
the various video frames (see Fig. 7B), they do not carry with them added metadata tags. The tags stay in 
the interface data file, set there by the designer not the user. As noted above, the Examiner also stated 
(see p. 1 1 of the final Office Action of September 4, 2009) that "a user indeed tags objects according to 
where the user places them in the display itself, for example, in a screen of a zoomed in image, the object 
of interest would be at the center of the screen, where the designer place[s] it, while other distant obj ects 
may be omitted". As alluded to above, moving or placing the user designated item (the person walking) to 
the center of the screen does not result in selected metadata tags being added to the user designated 
item (the person walking). Madrane's center of the screen does not represent any metadata tags, and 
thus moving or placing the walking person or any other item designated in Madrane' s center of the 
screen would not result in the addition of metadata tags to that item (page 17, last paragraph, page 18, 
first paragraph). 

Examiner disagrees. 

As to 6), as acknowledged by Appellant, a designer designates the two people walking toward 
each other, i.e., the objects, as interesting to a user, and accordingly, the designer designates them as 
extractable, therefore associating metadata tags with those objects. A user then is able to select the 
objects so that only the two objects are displayed. Thus, as acknowledged by Appellant, a user is able to 
perform this action "because of the metadata tags associated with those objects," and therefore, moving 
or placing the designer designate item to the center of the screen does indeed result in selected metadata 
tags being added to the designer designated item. Further, Appellant alleges that Examiner, in a previous 
Office Action, stated that a "user" tags objects according to where the user places them in the display 
itself. One of ordinary skill in the art would readily recognize that by "user" Examiner meant "designer." 



Appellant argues: 
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7) Moreover, even though an object displayed on a frame may have a metadata tag associated 
with it (for example, designating it as being of interest for the user, which designation is done by the 
designer not the user), this does not mean that when this object moves around in subsequent frames, 
additional metadata tags are added to the object depending on where the object is. In Madrane, a user 
cannot tag objects in a video frame, or tag selective video frames or tag a "root image". The only tagging 
disclosed in Madrane is done by the designer prior to the release of the video data to the user, and this 
tagging has nothing to do with the user moving various video objects around on the display (page 18, 
penultimate paragraph). 

Examiner disagrees. 

As to 7), as acknowledged by Appellant, Madrane teaches tagging a video prior the release of the 
video to the user. One of ordinary skill in the art would readily recognize that the tagging is done in 
association with an action to be performed, i.e., displaying only the objects selected, zooming in on an 
object. Madrane (fig. 10; col. 16, 1. 25-33) teaches that a designer moves various video objects around on 
the display in order to assign a particular action to a user selection. Thus, Madrane teaches that a 
designer can tag objects in a video frame, tag selective video frame, and tag a "root image." 

Appellant argues: 

8) The portion of Madrane cited by the Examiner as teaching this further limitation merely 
discloses storage of data in a local memory unit and has nothing to do with adding metadata tags based 
on the user moving the media object from a first location of the display to a second location in the display. 
See final Office Action at pages 4 and 6; and Madrane at column 20, lines 45-50. Indeed, the cited portion 
of Madrane does not even mention a display. As discussed above, Madrane does not teach moving a 
media object from a first region of the display to a second region of the display, while adding metadata 
tags to the object depending on the region the media object is. At most, Madrane teaches controlling the 
display of a sequence of video frames depending on a designation by the user (page 20, penultimate 
paragraph). 

Examiner disagrees. 
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As to 8), Examiner reiterates that references to specific columns, figures or lines should not be 
limiting in any way. The entire reference provides disclosure related to the claimed invention. As stated 
above in 7), Appellant acknowledges that Madrane teaches tagging a video prior the release of the video 
to the user. One of ordinary skill in the art would readily recognize that the tagging is done in association 
with an action to be performed, i.e., displaying only the objects selected, zooming in on an object. 
Madrane (fig. 10; col. 16, I. 25-33) teaches that a designer moves various video objects around on the 
display in order to assign a particular action to a user selection. Thus, Madrane teaches that a designer 
can tag objects in a video frame, tag selective video frame, and tag a "root image." 

Appellant argues: 

9) The above passage of Madrane merely discloses changing the size of the display image based 
on a user-designated viewing position (see, in particular, the bold faced portion of the above passage). 
There is absolutely no disclosure of the size of the display area allocated to each set of metadata tags 
proportional to the number of media objects portrayed therein (page 21 , antepenultimate paragraph) 

Examiner disagrees. 

As to 9), 

One of ordinary skill in the art would readily understand that Madrane teaches a user-designated 
viewing position to enable allocation in the display of each set of objects carrying metadata tags 
proportional to the number of media object portrayed therein. In other words, when a user designates a 
viewing position at a greater/lesser distance from the root image, a proportionally greater/lesser number 
of media objects are portrayed therein. 

(11) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the examiner in the Related Appeals 
and Interferences section of this examiner's answer. 
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For the above reasons, it is believed that the rejections should be sustained. 

Respectfully submitted, 

/Jordany Nunez/ 

Examiner, Art Unit 2175 

2/25/2010 

Conferees: 

/William L. Bashore/ 

Supervisory Patent Examiner, Art Unit 2175 



/Kieu Vu/ 

Supervisory Patent Examiner, Art Unit 2173 



